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Abstract 

This study examines the effect of training in think alouds on literal and 
higher-order reading comprehension. Thirty-two (n = 32) eighth grade 
students of English as a foreign language (EFL) participated in the study. 
The participants were randomly assigned to control and experimental 
conditions. Descriptive statistics, correlation coefficients, and a 
multivariate analysis of variance (MANOVA) test were conducted. The 
results indicated that think alouds is positively related to over all reading 
comprehension, critical, and interpretive comprehension as well as 
effective in improving critical comprehension. The results are discussed 
and recommendations for further research are suggested. 

Recently, reading research and practice have focused on enabling 
readers to become proficient and self-motivated readers who monitor their 
comprehension through the application of a range of metacognitive 
strategies. Metacognition is a higher order thinking process dealing with a) 
knowledge about cognition and b) regulation of cognition. Knowledge 
about cognition involves readers’ awareness of the existence and use of 
various strategies while reading. Meanwhile, regulation of cognition 
means that readers become able to evaluate, plan, and regulate reading 
activities (Anderson, Osborn, & Tierney, 1984; Cohen, 1998). 

Think alouds, defined as the conscious disclosure of thought 
processes while reading, has been proclaimed as an effective technique in 
helping readers acquire a variety of metacognitive comprehension 
strategies such as evaluating understanding, predicting and verifying, and 
self questioning before, during, and after reading (Bauman, Jones, & 
Kessell, 1993; Pressley & McDonald, 1997; Cohen, 1998; Pressley, 1998; 
Wade, Buxton, & Kelly, 1999). Likewise, previous research on the 
applications of think alouds procedures in teaching reading has indicated 
that this technique is a useful mechanism in describing and defining the 
thought processes of readers (Crain-Thoreson, McCledon-Magurson, & 

Vol 27, No. 3, Mar 2004 



50 


Educational Research Quarterly 


2004 


Lippman, 1997) as well as enhancing learners’ self-regulated learning 
(Zimmerman, Bonner, & Kovach, 1996). Similarly, Smith (1991) 
concluded, based on empirical evidence, that think alouds sheds light on 
readers’ comprehension processes as they read written discourse. 

In addition, several studies have reported empirical evidence that 
training in think alouds enables readers to acquire a wide-range of 
strategies that enhance comprehension and enable them to overcome text 
difficulties (Bauman, Jones, & Kessell, 1993). Furthermore, Silven & 
Vauras (1992) and Gordon &Day (1996) reported that training in think 
alouds improves gist comprehension and main idea identification. 

The preceding review suggests that training readers in the 
application of think alouds strategies sheds light on their thought processes 
during reading as well as improves their comprehension of the gist and 
main ideas of written discourse. Furthermore, educators such as Carrell 
(1989) and Young (1993) have called for extending the application of 
think alouds and other metacognitive strategies into the domain of second 
and foreign language reading instruction. However, it remains unclear 
whether think alouds improves all types of literal and higher-order 
comprehension, especially when reading materials in a second or foreign 
language. 

Therefore, the purpose of the present study is to examine the 
effect of training in think alouds on improving literal and higher-order 
reading comprehension of readers of English as a foreign language as 
described by Burns, Roe, & Ross (1999). These researchers defined literal 
comprehension as ability to understand directly stated ideas in the text 
thereby follow written directions, recognize details and sequence, and 
understand cause-effect relationships. Meanwhile, higher-order 
comprehension requires interpretation, analysis, and synthesis of 
information and involves the three types of interpretive, critical, and 
evaluative comprehension. 

Burns, Roe, & Ross (1999) define interpretive comprehension as 
reading between the lines and making inference in order to derive ideas 
that are not directly stated in the text. As such, interpretive comprehension 
includes inference about main ideas, implied cause-effect relationships, 
and understanding pronoun and adverb referents. Interpretive 
comprehension also includes detecting the mood of a passage and the 
author’s purpose in writing it, drawing conclusions, and interpreting 
figurative language. Meanwhile, critical comprehension involves 
evaluating written material by comparing the ideas presented in the 
material with known standards and drawing conclusions about their 
accuracy, appropriateness, and timeliness. As such, critical comprehension 
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depends on both literal and interpretive comprehension and on readers’ 
ability to grasp implied ideas. Finally, creative comprehension involves 
going beyond the material presented by the author to produce new ideas 
and apply what has been read. 

Specifically, the study addressed the following questions: 

1. What is the relationship between mastery of think alouds 
procedures and literal and higher-order reading comprehension? 

2. Is training in think alouds more effective than regular instruction 
in improving literal and higher-order reading comprehension? 

Method 

Participants 

The participants were 32 eighth-grade learners of English as a 
foreign language (EFL) enrolled in a private school in a Middle Eastern 
country. The participants were randomly assigned to control and 
experimental conditions. The control group included 16 participants and 
received regular reading instruction; where as, the experimental group 
included 16 students and received instruction in the applications of think 
alouds strategies in reading the same material read by the control group. 
The age of the participants ranged from 12-13 years. 

Instruments 

A reading comprehension posttest was administered to all 
participants in the control and experimental group at the conclusion of the 
study. This test was specifically designed for the purpose of present study 
and intended to measure the four types of reading comprehension under 
investigation (literal, interpretive, critical, evaluative). The content validity 
of the test was ensured by using a specification table as suggested by Gay 
(1996). The specifications covered the reading aspects involved in the four 
types of comprehension and were matched to specific test items that 
measure these aspects. The test consisted of 30 items: 7 items measure 
literal, 8 items interpretive, 5 items critical, and 10 items creative 
comprehension. Finally, the posttest was based a reading selection titled 
“The Lady or the Tiger” and was not read before by the participants in the 
study. 

In addition, a think alouds checklist was used in the study and 
aimed at providing additional practice in the use of think alouds procedure 
as well as analyze the interview protocols of the students in the 
experimental group. The checklist focused on determining the frequency 
of using the strategies involved in the think alouds procedure and included 
a five -point scale designed to determine the mastery level of the think 
alouds technique. 

Study Design 
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The study employed an experimental posttest-only control group 
design. The experimental group received training in the application of 
think alouds strategies as they read selections from the regular academic 
program of the participants. These strategies included predicting, 
picturing, comparing, identifying problems, and using fix-up measures. 
Predicting involved using illustrations and titles to predict what the text 
being read might be about. Picturing entailed visualizing characters and 
settings. Comparing focused on story events and life experiences. 
Identifying problems covered vocabulary, misinterpretation, and 
misunderstanding of written material. Finally, the fix-up measures 
included rereading, reading on, self-questioning, predicting and verifying, 
making reflections, asking whether what is read makes sense, and 
retelling. 

The control group read the same material read by the 
experimental group according to the procedures suggested in their 
textbook. These procedures focused on prediction, teacher explanation, 
question/answer, and vocabulary instruction. In order to control for 
validity threats, both the experimental and control groups were taught by 
the same teacher who did not have any pre-perceived desirability of the 
effects of the treatment in improving comprehension. Furthermore, all 
instructional conditions were identical for both groups except for the 
treatment. More specifically, both groups followed an identical 
instructional scope and sequence during the first three phases of the study. 
However, individual interviews were only conducted with the participants 
in the experimental group in the last phase of the study as explained 
below. 

The study was implemented in four phases. Phase I focused on 
training the teacher who participated in the study in the application of the 
think alouds procedure. This phase lasted for one week, following which 
Phase II started and continued for four weeks. During this phase, the 
participants in the experimental group received instruction and practice in 
applying the procedures of the think alouds strategy as they read written 
material. The first two weeks of phase II were focused on individual and 
pair practice and the last two weeks involved further demonstrations of the 
think alouds procedures by one of the researchers, the teacher, and 
individual participants. Phase III lasted for one day during which all the 
participants in the study were administered the reading comprehension 
posttest. Finally, phase IV lasted for one week and involved conducting 
individual interviews with the participants in the experimental group in 
order to determine their mastery level of the think alouds procedures. Each 
interview lasted for about twenty minutes during which each participant 
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read a short selection and applied the procedures of the think alouds 
strategy. The verbal protocols of the participants were tape-recorded and 
analyzed in order to determine their mastery level of the think alouds 
procedures. 


Data Analysis 

Two independent raters assessed the participants’ performance on 
the reading posttest. The inter-rater reliabilities were as follows: 1) overall 
reading comprehension score r = 0.98, literal comprehension r = 1.0, 
interpretive comprehension r = 0.76, critical comprehension r = 0.89, and 
creative comprehension r = 1.0. In addition, the raters scored the verbal 
protocols of the participants in the interviews to determine their mastery 
levels of the think alouds procedures and achieved an inter-rater reliability 
of r = .92. 

Descriptive statistics (Means and Standard Deviations) were 
computed and Pearson Product-Moment Correlation coefficients were 
computed in order to address the first question in the study regarding the 
relationship between the mastery levels of think-alouds and the different 
types of reading comprehension. 

In addition, a Multivariate Analysis of Variance (MANOVA) test 
was run in order to address the second question raised in the study 
regarding the effect of training in think-alouds on the different types of 
reading comprehension. The treatment condition (experimental versus 
control) was used as independent variable (Factor) and the levels of 
reading comprehension on the posttest (literal, interpretive, critical, and 
creative) were used as dependent variables. 

Results 

Correlation Analysis 

The correlation analysis indicated a significant relationship 
between mastery level of think alouds and overall reading comprehension, 
r = 0.73, P < 0.01. Likewise, there was a significant relationship between 
mastery level of think-alouds and critical comprehension, r = 0.53, P < 
0.05 and a significant relationship between mastery level of think alouds 
and interpretive comprehension, r = 0.72, P<0.01. 
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Table 1: Correlation Coefficients Between Mastery Level Scores of 


Think-AIoud and Reading Comprehension 


Overall Score 

Literal 

Interpretive 

Critical 

Creative 

Think- Alouds 0.73** 

Score 

0.18 

0.72** 

0.53* 

0.36 

** P <0.01 * P <0.05 


However, the analysis did not reveal a significant relationship between 
mastery level of think alouds and creative comprehension, r = 0.36, P > 
0.05 and between mastery level of think alouds and literal comprehension, 
r = 0.18, P > 0.05. (See Table 1). 

MANOVA Analysis by Treatment 

Table 2 presents a summary of descriptive statistics (means and 
standard deviations) by treatment, overall comprehension, literal 
comprehension, and higher-order comprehension. The results of the 
multivariate analysis of variance MANOVA test on the comprehension 
scores of the experimental and control group are shown in Table 3 and 
reveal the following: 

Table 2: Summary of Means and Standard Deviations of Posttest 
Scores by Treatment 




Overall 

Literal 

Interpret 

Critical 

Creative 



Score 




ive 
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M 

S 
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S 
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S 
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S 
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S 
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D 

Ex 

1 

15 

2. 

4. 

0 . 

3. 

1 . 

3. 

0 . 

4. 

0 . 

P 

6 

.8 

52 

28 

89 

90 

06 

68 

98 

06 

79 



7 










Cnt 

1 

14 

2. 

4. 

0 . 

3. 

0 . 

2. 

1 . 

4. 

0 . 


6 

.8 

36 

78 

57 

59 

96 

40 

33 

12 

61 
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Note. Exp. = experimental; Cnt. = control; Trt. = treatment; & 
overall score = overall reading comprehension posttest score 
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First, the analysis indicates a significant difference between the 
experimental and control group by treatment F (5, 26) = 3.17, P < 0.05. 
Second, the univariate analysis of variance shows a significant difference 
in literal comprehension in favor of the control group F ( 1 , 32) = 3 . 1 0, P < 
0.01; the mean score of the experimental group was 4.28 (SD = 0.89), and 
the mean score of the control group was 4.78 (SD = 0.75). However, there 
is a significant difference with regard to critical comprehension in favor of 
the experimental group F (1, 32) = 9.59, P < 0. 01; the mean score of the 
experimental group was 3. 68 (SD = 0.98), and the mean score of the 
control group was 2.40 (SD = 1.33). Finally, there are no significant 
differences at the P < 0.05 level between the control and experimental 
group with regard to overall reading comprehension F (1, 32) = 1. 34, P = 
0. 25, interpretive comprehension F (1, 32) = 0. 75, P = 0.39, and finally 
creative comprehension F (1, 32) = 0. 06, P = 0. 80. 


Table 3: Multivariate Analysis and Univariate Analysis of Variance 
MANOVA by Treatment 



Multivatiate ANOVA (a) 

Universe ANOVA (b) 



Score 

F Overall 

Score 

Literal 

Interpretive 

Critical 

Creative 

Treatment 

3.17** F 1.34 

3.10* 

0.75 

9.59*** 

0.06 

a dfs. = (5,26) 

b dfs. = (1.32) *P<0./1 

**P<0.05 

1 ***P<0.01 




Discussion 

This study was conducted to investigate the connection between 
think alouds and literal and higher-order comprehension and the effect of 
training in think-alouds on improving the reading comprehension of eighth 
grade EFL students. The first question was about whether the mastery 
levels of think-alouds are related to readers’ literal and higher-order and 
reading comprehension. The results of the correlational analysis indicated 
that there was a significant relationship between mastery levels of think 
alouds and overall reading comprehension, critical comprehension, and 
interpretive comprehension. This underscores the role of metacognitive 
strategies such as think alouds as a determinant factor of reading 
comprehension. The pedagogical implications call for deliberate 
instruction in the applications of think alouds strategies in order to 
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improve readers’ comprehension of written discourse as suggested by 
Carrell (1989) and Young (1993). 

The second question was about whether training in think alouds 
would affect reading comprehension as measured according to the 
different types of reading comprehension (literal, interpretive, critical, 
creative). The results indicated that there was a significant difference 
between the experimental and the control groups by treatment. More 
specifically, the univariate analysis indicated that students in the 
experimental group scored higher on critical comprehension whereas 
students in the control group scored higher on literal comprehension. This 
contradicts the findings of Silven & Vauras (1992) and those of Gordon & 
Day (1996) who reported that training in think alouds improves gist 
comprehension and main idea identification. One possible explanation of 
these findings is that the think alouds strategies promote readers’ 
reflection skills on their comprehension, thereby improves understanding 
of reading aspects that require reflection and monitoring of own learning. 

On the other hand, the results showed that students in the control 
group scored higher on literal comprehension than their counterparts in the 
experimental group. Literal comprehension requires recognition and 
understanding of information and cause-effect details that are directly 
stated in the text as well as placing story events in order. As such, it could 
be that reading instruction that involves teacher explanation and 
question/answer exercises improved the literal comprehension of the 
control group. This suggests that reading instruction and teaching practices 
should be planned and determined based on the reading program 
objectives and instructional needs of the learners, which calls for an 
eclectic approach that utilizes a variety of instructional techniques that 
range from teacher explanation and question/answer to metacognitive 
strategies. 

Finally, further research is needed in order to determine the 
relative role of metacognition as a determinant of literal and higher-order 
comprehension. It is also equally important to investigate the effect of 
training in think alouds and other metacognitive strategies on reading 
comprehension in various cultural and linguistic contexts. 
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